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Abstract 

The use of data analysis software in the process of qualitative research has become 
commonplace. Use of such software has clear technological implications in terms of knowledge 
which must be acquired by researchers in order to use the software. In addition, computer 
hardware and software have come to be seen as necessary for the conduct of qualitative research. 
We believe that in addition to technological implications, there are methodological and analytical 
consequences for the use of qualitative data analysis software. In this paper, we present 
perspectives gained from the conduct of three qualitative research studies. While the studies 
differ in their theoretical frame, areas of interest, and scope, their common employment of a 
particular data analysis software package give us a means by which to explore issues which result 
from the use of the software. 
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Computers and specialized software for data analysis have had a far-reaching impact on 
qualitative research. Richards & Richards (Richards & Richards, 1994) call the potential 
implications of the use of qualitative data analysis software “dramatic” (p. 445). Bogdan & 
Biklen (Bogdan & Biklen, 1992) call the use of computers to analyze qualitative data “The 
greatest change in research technology in the last decade-and-a-half’ (p. 181). This change, 
however, is seen by Bogdan and Biklen as “more technical than conceptual” (p. 27). While 
technological affordances and impediments may be the most conspicuous issues, we’ believe that 
the conceptual issues involved in using software to aid in data analysis are worth consideration. 
Unpacking these issues may help future qualitative researchers as they make study design 
choices and decisions regarding the use of computer software as an aid to their analysis. 

To explore the conceptual consequences of the use of data analysis software, we present 
perspectives from three current qualitative studies which have in common their use of a particular 
data analysis software: QSR’s NUDIST. Our interest is not centered on technological concerns; 
we focus instead on methodological and analytical issues raised by the affordances and 
constraints of the software used in the data analysis process. 

In this paper, we write about “methodological” and “analytical” issues. For our purposes, 
methodological issues are those which affect the design of a study or which affect how data 
collection is carried out. Analytical issues are those which impact the analysis phase of a study, 
after data has been collected; in the methodologies employed in the three studies, analysis and 
writing phases of the research process were somewhat permeable. More will be written about 
this later in the paper, but for definitional purposes, analytical issues in this paper may occur 
during the analysis of data and/or during the writing of study results. 

Affordances, as we use the term here, are facets of the analysis software which encourage certain 
courses of action. This definition grows out of the ecological psychology use of the term (see 
(Gaver, 1991), but in this paper we consider the consequences of affordances in addition to the 
affordances themselves. 

Data representation, like software affordances, has methodological effects (Larkin & Simon, 

1987). The ability of analysis software to represent data in certain forms causes the researcher to 
structure her methods along lines allowed by the representational capacities available. 
Additionally, some types of results are more easily represented and thus more easily seen than 
other types; use of the software may highlight trends which would otherwise be considered 
inconsequential and may hide trends which otherwise might be quite important. Thus, analysis 
which is supported by a particular software tool differs both from analysis not supported by 
software and by analysis supported by other tools. 



' In sections which refer to issues common to the three studies, “we” refers to this paper’s three authors. “I” is used 
in the sections describing each individual study and refers to the study author. 
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This paper focuses on the methodological and analytical consequences of the use of NUDIST in 
three educational research studies. For each study, we discuss the choices which data analysis 
software allows and encourages, commenting on the methodological and analytical effects of these 
choices. The variety of conceptual frames and data types represented by the three studies 
combined with the common use of the same analysis software allows ample opportunity for 
comparison and contrast among the studies. Data structure, coding hierarchy, flexibility in coding, 
and reporting capacities are examined in each study, yielding a depiction of the methodological 
and analytical influences of this particular qualitative data analysis software tool. 

QSR NUDIST Described 

Before describing the studies and considering their respective methodological and analytical 
themes, this section briefly describes QSR NUDIST, the software we used for data management 
and analysis. This brief overview is intended to familiarize readers with the basic capabilities and 
modes of operation of the software. 

At the broadest level of organization, all of the data in a particular study are contained in a 
“Project”. Within the project are the raw documents which comprise the data, the coding 
structure, and assorted notes and memos created by the researcher in the process of managing the 
data. In the subsections below, we illustrate the organizational components and metaphors used 
by NUDIST in constructing its projects. 

Document Handling 

nudist’s ability to allow a researcher to manage large numbers of documents is one of its prime 
attractions. Figure 1 below shows the simple system used to display the documents within the 
NUDIST project. Over three hundred pages of transcripts are represented by the list of 
documents at the left side of the window in the illustration below. NUDIST combines the 
accessibility to text of a word processing program with the data handling capacity of a database 
program, as well as some of the reporting features of a spreadsheet. 
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Figure 1 : Document Explorer Window 



The seventeen documents which 
comprise the data for study 1 
are listed here. 



□ 



Document Explorer 






AcHTon Sims 97/09. 
Aaron Sims 97/12/5 
A^ron Sims folio... 
Amira Hendry 97/... 
Amsra Hendry 97/... 
Amsra Hendry foil... 
Barbara Greenwa... 
Barbara Greenwa... 
Barbara Greenwa... 
Beth Soles 97/09/ 
Beth Soles 98/01/. 



Online Document: 

|B airbara Gieenvalt 97/ 12/17 
Header: 



*Interviev Date December 17, 1997 
*Interviever Jon Margerum-Leys 
^Participant Barbaia Greenvalt (A pseudon>7n) 
♦Interviev location Prechter Conference Room 
*Inteiviev time 3:00 P.M. 



Make Report 



Memo i 


Browse 


Propertied 


Close 



The researcher determines the kind of header 
information for each document in the system. In this 
example, the date, time, place, and participants are 
listed for the selected interview. 



The Node System 



Text which is entered into NUDIST can be organized and analyzed through the creation and use 
of a coding system; this coding is one of NUDIST’s primary functions. The coding system, in 
turn, is structured through a system of nodes. Each code, whether it is thematic (relating to the 
content of the data) or stmctural (relating to the stmcture of the data) is assigned to a node on the 
coding tree. These nodes can be hierarchically arranged, as shown in Figure 2 below. 
Alternatively, nodes can be simply listed in a non-hierarchical stmcture such that all nodes are at 
one level. In addition to showing the structure of the nodes, the node explorer shown in Figure 2 
gives some basic information about the definition of the selected node (“social dynamics of 
computers” in this case) and the amount of text in the project which has been assigned to that 
node. 
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Figure 2: Node Tree 



The index tree gives a visual representation of 
the coding hierarchy. Note that this figure 
shows three levels of the tree, displayed in 
outline form. 



Coding status shows how many documents 
contain some text which is coded with the 
selected node, as well as how many total text 
units are assigned that code. 



\ 



Free Nodes ( 8 } 

Index Tree Root [ 107 ] 

[«)— 1 BacKgrourid Information 















3 Computers as used by teachers • 

— 1 Progressive teachers use computers 
g- 2 As a teacher tool, computers are useful for 

1 Making presentations 

2 Keeping Grades 

— 3 Tracking student data 
— A Information gathering for teacher 

— 5 Creating resources for students 
L— 3 Teachers need training 
A Computers and curriculum 

5 \ Computers as used by students 

6 ! Learning about computers (participants) 

7 ! Kno vledge about computers 

8 ! Availability of computers 
Text Searches [ 13 ) 

Index Searches I 6 1 . ... 



Node: 



ji' Social 







computers 



Coding Status: 



|17 documents, 708 text-units. 



Definition: 



In this category, participants express their beliefs about hov 
computers affwt social interactions, vithin small groups and in 
society at large. 



The node definition gives a brief 
description of the kind of text which is 
assigned to the selected node. 



Working With Individual Documents 



Figure 3 below shows an individual document from study 1 of this paper in the process of being 
browsed. In the figure, the text has been coded, though the coding is not immediately visible. 

The full text of the interview can be seen and the coding of any particular piece of text is 
accessible in a separate small vdndow. In the coding process, any piece of text can be assigned to 
one or more codes, each of which is a node in the coding tree. 

Note that the selected line of text in the figure below is assigned four codes; ‘Background 
information,’ ‘During Fall 97,’ ‘Second Round,’ and ‘Tally of Coded Nodes.’ The first two 
codes are thematic, dealing with the content of the text. Text coded as ‘Background information’ 
concerns statements by a participant concerning her background; ‘During Fall 97’ is the code for 
statements about a participants’ experiences during that particular period of time. The second 
two codes are not thematic; rather, they are used structurally for data management. ‘Second 
round’ is a code assigned to all data from the second round of interviews. By assigning this code, 
the researcher could quickly separate out data from this round; combining thematic and 
management codes allows the researcher to find, for example, all statements made in round two 
referring to background information. ‘Tally of Coded Nodes,’ the final code shown below, was a 
very general code assigned to all text which had any coding assigned to it. This allowed the 
researcher to quickly calculate the percentage of data captured by the coding system. 
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Figure 3: Browsing and Coding 



The full text of an interview from study 1 is shown. In this example, 
each line of text forms a unit which can be coded separately. 






Free Nodes { 8 ) 

Index Tree Root [ 107 ) 
di— 1 Background Informat 



9“ 3 Computers as used b) 
1 Progressive teache 
0“ 2 As a teacher tool, ( 
— I Maldng present 

— 2 Keeping Grade: 

— 3 Tracking studei 

— 4 Information gat 

— 5 Creatiiig rasoui 

3 Teachers need trail 

4 Computers and cunic 

5 ! Computers as used 

6 ! Learning about comp 

7 ! Knovledge about coi 

8 I Availability of compu 



Make Report' 



Mem 



Aaron Sims 97/09.. 
Aaron Sims 97/12/5 
Aaron Sims folio... 
Amira Hendry 97/... 
Amira Hendry 97/... 
Amira Hendryfoll... 
Barbara Greenwa... 
Barbara Greenwa 
Barbara Greenwa... 

Rot-h Cntoc 07 /no/ 



_J L 



i 1^ 



Document 3rows/r: 'Serbara Greenwalt 97/12/87’ : 82 - 82 



Barbara: Thot, I mean, that’s a yse for it. At least it gets used every 

day. And I ’n> like ‘okay, so thays a use for it', but other than that, 
that's the only time it goes ouy And the things that do get used a lot 
OTQ, uie have ifuo UCR's and TU'^in there [the social studies office]. 

And they get pulled out. And They get used on an almost daily 
basis. Urn, which seems odd^ J^cause everyone has a TU in their 
classroom, but then I guess ^/^ o one has UCR's hooked up to them. 

Uhich seems kind of, you knVT disappointing on the other side. 

Urn, 

Jon: Uhat would they do with the TU‘s without a UCR? 

Barbara: They’re, they’re Channel One TU’s. I guess like no one. I 
mean, you're supposed to have this opportunity, like, I doubt if I ‘II 
usG them either, but, they have, you know, different programs that 
they run during the day, at different times. Vou have a listing for 
supposedly that you can get, to see if you wanna have this show or 

that show, and they can set it up to have that. You know, for your 

classroom. 

Jon: Does tCT’s name] use that? 

Barbara: Mo. Mo, not at all. Ue have a buttload of videotapes. 

Uhich, I am fairly poll acquainted oith, since I've been, 

categorizing and moving them. ,JUn — tbc>cr*‘«^ a Int nf 

different stuff. And some of “ Examine Coding 

them look like really cool r:? ^ — 






idex 



t 53 



C^iiine Document: 



I Barbara Green valt 97/12/17 



Header: 



Inteiviev Date December 17, 1997 
*Inteivie'&-er Jon Margerum-Leys 
* Participant Barbara Greeni/alt (A p: 
*Interviev location Prechter Confere 
“^‘Interviev time 3:00 P.M. 



Background Informatior 
During Fall 97 
Second round 
Tally of Coded Nodes 



Node: 



Definition: 



Daete References 



Browse j ( Close j 



The particular codes assigned to the 
selected line above are shown here. 



Reporting Coding Results 



A variety of flexible reports can be generated from NUDIST at any point in the coding process. 
From a methodological standpoint, this encourages an iterative approach to the coding process; 
since reports can be quickly generated, it is easy for a researcher to check on the nature and 
effectiveness of her coding structure repeatedly during the coding process. 

Reports can be created from two main perspectives: Code centered and document centered. 
Document centered reports, as shown in Figure 4, allow an overview of the thematic contents of 
a particular piece of data. Code centered reports, shown in Figure 5, allow views of a theme 
across multiple pieces of data. 
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Figure 4: Document Report 



m Jeport on ‘Barbara < 



Q.S.R. NUD*IST Power version, revision 4.0. 

Licensee: The University of Michigan. 

PROJECT: Master project. User Jon Margerum-Leys, 3:00 pm, Apr 7, 1999. 



+++ ON-LINE DOCUMENT: Barbara Greenwalt 97/09/22 

++ Units: 45-79 
351-456 459-514 

<1 1 1 > 

++ Units: 155-223 
< 1121 ) 

++ Units: 284-314 
<1 1 3) 

++ Units: 459-5 14 
<1 2 1 ) 

++ Units: 228-234 
<1 22 1 ) 

+f Units: 45-67 
(1 2 4) 

Units: 73-79 
<1 2 6 ) 

++ Units: 55-79 
< 2 ) 

++ Units: 327-333 
<2 2 ) 

++ Units: 672-677 
<2 2 7) 

++ Uni ts : 672-677 
<2 6 > 

++ Units: 839-842 
<2 7) 

++ Units: 327-333 

RlWlf" 






/Background I n f or ma t i on 

81-116 120-138 155-223 228-239 244-264 284-314 327-341 

520-537 707-723 

/Bcackgnound I nfonmation/ I n Technology/ I n K12 Education 

234-239 244-264 707-723 

/Background Information/In Techno logy /Post secondary education or as a teacher /Prior 
327-34 1 

/Background In formation/ In Technology/ In the work arena 
520-537 

/Background Information/Outside technology/ln K12 Education 

/Background Information/Outside techno logy /Postsecondary education or as a teacher /I 
351-456 

/Background Information/Outside technology/Chi Id non-education 

/Background Information/Outside technology/Motivation for teaching or language arts 
81-116 120-138 

/Social dynamics of computers 

672-677 764-778 783-791 800-805 839-842 844-845 931-941 

/Social dynamics of computers /Has an effect on society 

/Social dynamics of computers/Hos an effect on society /Has parallels outside techno 

/Social dynamics of computers/No negatives to use of computers 
844-845 

/Social dynamics of computers/Negatives to using computers 
764-778 783-79 1 800-805 93 1 -94 1 






In the illustration above, the codes assigned to a particular document are shown. By referring to 
the node tree, the researcher can see which themes can be found in this interview. 

Figure 5: Code Report 



Report on C4 2)1 



same time, I don't think that, that math students could benefit any 
more than, than students in a different, subject area. Urn, I guess 
++ Text units 331-334: 

absolutely it can, it has a place in that classroom. And, and urn, 
yeah, I don't know, Urn, I guess I, I guess I can only see it, yeah, I 

can't see any sort of like, hierarchy where it would be, more useful 

in one or the other. I don't know if that jibes with what I said last 
++++++++++++++++++++++++++++++++++++++++ 

+++ ON-LINE DOCUMENT: Amira Hendry 97/12/12 

+++ Retrieval for this document: 8 units out of 495, = 1.63? 

++ Text units 287-294: 

Jon; No, don't be, that's fine. Urn, thinking about the 
subject areas, urn, language arts and uh, social studies 
and math, and English, and science, and media, and phy 
ed, music, and art, and German and Japanese and all 
that, urn, is technology best suited for certain subject 
areas as opposed to others? 

Aroiro; No, I pretty much think it's fitting for all of them. Uh, 
even math. 

+++ ON-LINE DOCUMENT: Barbara Greenwalt 97/09/22 

+++ Retrieval for this document: 6 units out of 1092, = 0.553? 

Text units 683-690: 

Barbara: I think it depends. I think some subject areas you hck/e to 

be more creative. 

Jon: Like which ones? 

Barbara: Urn... I ike, actually, in some ways, you have to be more 
creative in social studies. Because it's very easy to say you can just 
use it for research, but, is that it? 
++++++++++++++++++++++++++++++++++++++++ 

+++ ON-LINE DOCUMENT: Barbara Greenwalt 97/12/17 

Retrieval for this document: 4 units out of 654, = 0.613? 
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The report illustrated above shows the percent of text coded for the particular code being 
reported. For example, the interview “Amira Hendry^ 97/12/12” has 1 .6% of the interview 
assigned to this code. Following the percentage is the full text coded, along with the line numbers 
at which the text appears. 

Ease ofU.se Concerns 

NUDIST offers a powerful, flexible system for qualitative data analysis. Its capabilities facilitate 
an iterative, bottom-up approach to coding textual data. However, the learning curve for this 
software is quite steep. Even when the program has been learned thoroughly, its counterintuitive 
modes of operation can be frustrating. Two of this paper’s authors (Margerum-Leys and 
Kupperman) hold Master’s degrees in educational technology; both consider themselves 
reasonably skilled technologists. Kupperman has taken part in a day-long seminar on NUDIST. 
Still, using NUDIST remains a challenge for all three of this paper’s authors. 

Is nudist’s difficulty-of-use an analytical concern? Perhaps. It might be argued that 
ineffective use of NUDIST could lead to problematic data analysis. Incorrect coding of a data set 
or misreading of a NUDIST report could lead to erroneous research conclusions. For example, 
consider Figure 4 above. It is not easy to interpret the results of the report shown; the format of 
the report is not conducive to easy interpretation, and no graphic representation of the data is 
available. It is conceivable that a researcher who had difficulty generating the appropriate report, 
or who was unable to interpret the reports created by NUDIST, could reach conclusions not 
warranted by her data. This would be true of any researcher using any analysis system; 
however, NUDIST’ s difficulty of use highlights this potential problem. 

Conclusion of This Section 

Though we have reservations about the difficulty of learning and using NUDIST and believe that 
these difficulties may have analytical antecedents, NUDIST offers affordances which justify its 
use. Its ability to manage large amoimts of data while retaining access to the data itself make it a 
useful tool. The sections above give the reader a sense of the capabilities and modes of operation 
offered by NUDIST. We use this description as a means to help the reader visualize the 
relevance of the methodological and analytical issues explored in the sections below. 

Study Descriptions and Methodological and Analytical Issues Addressed 



In the sections below, we describe and discuss the three studies which contribute to this paper. 
For each, we describe the research setting, methodology, and analysis used. These descriptions 
serve to create a picture of the studies as a basis for describing the analytical issues entailed in 
using NUDIST for data analysis. 




^ A pseudonym 
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Study 1: Teacher Education Students’ Beliefs About Technology 
Study author: Jon Margerum-Leys 

Study Description 

This interview-based study examined the beliefs of nine teacher education students at a major 
research university. The participants were members of a year-long field-based teacher 
preparation cohort, in which teacher education students earn a Master’s degree in education and a 
teaching certificate. A complete description of this study’s participants and methods, along with 
the study results, appeared as an AERA poster session (Margerum-Leys & Marx, 1999). 

The interviews which comprised the study data were loosely structured around protocols 
consisting of approximately one dozen questions, with follow-up probes for each question. 

While the same protocol was followed for each interview, conversations varied somewhat due to 
the individual interests and experiences of the participants. 

Interview times ranged between 35 minutes and 75 minutes. Audio tapes were transcribed word 
for word, including non-word utterances. Transcribed interviews were coded using verbal analysis 
coding, an analysis method suggested by Chi (1997). 

Verbal analysis coding is a method by which a researcher’s subjective impression of the data is 
formalized, combined with an analysis which examines the frequency with which items identified 
as fitting the formal pattern occur. Creation of the coding and coding categories is “bottom-up,” 
arising from the data rather than a theoretical construct. 

The verbal analysis process yielded eight coding categories: Background information, social 
dynamics of computer-based technology, computers and curriculum, computers as used by 
teachers, computers as used by students, learning about computer-based technology, knowledge 
of computer-based technology, and availability of technology. The categories in turn were parsed 
into 84 distinct codes. In this study, the complex coding tree represents my subjective picture of 
the data. NUDIST was used to map the 13,000 lines of transcribed material onto the coding tree. 
Using this structure, the data were examined for patterns within and across codes and categories, 
participants and interview sets. 

Methodological and Analytical Issues 

For this study, NUDIST supported an iterative, bottom-up approach to data analysis. While it 
is possible to accomplish bottom-up coding without the use of data analysis software, the ability 
to create complex coding structures which could be easily modified encouraged me to pursue 
coding in which the structure arose from the data. Several issues follow from this: 



ERIC 
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Consideration of analysis capabilities during study design 

When designing the study, 1 considered among surveys, structured interviews, and semi- 
structured interviews as sources of data. Surveys and structured interviews offered a built-in 
starting place for a coding structure; a code could be established for each survey item or interview 
question. Comparison between participants would be straightforward, with participants’ 
answers matched on an item-by-item basis. However, 1 felt that a semi-structured interview 
would give participants a greater chance to express themselves. In turn, if the questions 1 asked 
were less structured and avoided leading the participants in too-specific directions, 1 would have 
more basis for a claim that interview responses reflected participants’ beliefs. 

nudist’s ability to flexibly facilitate thematic and structural coding was an influence in my 
selection of semi-structured interviews as a data source. Creation of a data structure was not 
dependant on preset items. The ability to handle a large set of documents, searching and 
reporting quickly across the set, persuaded me that it would be feasible to draw structure from an 
ill-structured set of interviews. 

Calling attention to data trends 

As 1 generated codes and coded the interview data, 1 was able to repeatedly explore how often 
particular codes occurred within the data. In some cases, this led to alteration or rejection of a 
particular code. For example, codes which occurred infrequently or which occurred only within a 
particular interview were examined for combination with similar codes or for rejection from the 
coding structure. 

There were instances in which codes arising from the data came to my attention unexpectedly. In 
one case, 1 discovered that all participants had mentioned access to technology as a critical 
element in implementation. None of the interview questions had queried participants on this area 
and 1 had not thought it would occur in conversation. However, exploring the developing codes 
brought to my attention that the topic occurred in every interview. Without the ability to 
quickly and flexibly examine my developing data structure, this data trend might have gone 
unnoticed. 

With eighty-four codes in eight distinct categories, tracking the relative counts and percentages of 
each code and category was a formidable challenge. The capacity to do so allowed me to carefully 
look for trends in the data. While the data in this study were not quantitative, tracking the counts 
and percentages of each code gave me a numeric sense of a very complex data structure. 

Switching back and forth between the quantitative nature of counts and percentages and the 
qualitative data itself allowed a rich combination of perspectives on the data. 

Counts and percentages as a means to justify reporting trends 

Verbal analysis coding (Chi, 1997), the analysis system used in this study, brings some 
quantitative justification to qualitative data. Codes which occur across several participants, recur 
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from interview to interview with a particular participant, or which account for a significant 
percentage of the data, can be justifiably deemed worthy of reporting. 

Consider the example above, in which participants talked about access to technology as a key 
element of their beliefs about how technology is and should be used in education. By using 
NUDIST, I was able to confirm that all of the participants had alluded to this issue, and that for 
several, it was a major focus of their beliefs about technology. Access to technology as an issue 
accounted for an average of 4.8% (range 0.38%- 13%) of the interviews. The ability to have some 
justification for inclusion of a category such as this, combined with the ability to closely examine 
incidences of participants talking on a particular subject, gave me confidence in my analysis of 
the data. 

Making the data structure explicit 

Within the NUDIST software, it is possible to see a visual display of the developing coding tree. 
In concert with Inspiration software’s Inspiration, NUDIST also supports the creation of more 
easily-grasped concept maps of the coding structure. Appendix A contains the concept maps for 
the eight data categories in this study. Creating rough drafts of these maps to represent early 
iterations of the coding structure was the work of minutes. For me, these early maps represented 
an epiphany in the analysis portion of this study. Like the blind men and the elephant (Saxe, 
1968), I had seen portions of the data but had never been able to step back and view the big 
picture. Multiple perspectives on data are invaluable in the analysis process. With the visual 
array provided by the maps of the coding structure, I had three perspectives: The mass of 
qualitative data itself, in which the participants’ voices could be heard; the quantitative look 
provided by the counts and percentages reporting; and an impression of the skeleton of the body 
of data presented by the maps. This last was invaluable in allowing me to see the shape of the 
data as it developed. 

Potential criticism of over-rating the validity of the structure 

NUDIST allowed me to create a complex analysis structure which is nonetheless comprehensible. 
This structure seems to me to accurately represent the data. As I see it, the coding arose directly 
from the data and the structure exists as it does as a result of the nature of the underlying data. It 
is conceivable, though, that I have created a castle made of straw. Straw, left to its own devices, 
would never resemble a castle. Perhaps the data in the study under consideration do not really 
coalesce into the structure I have created. 

When using NUDIST, it is easy to be lulled into believing that one’s own initial coding is correct. 
Once coded, data can be mined for an array of very convincing reports and maps. These appear 
to validate the researcher’s analysis. At the level of counts and percentages and visual maps, 
though, what is being reported is not the data itself, but the subjective structure created by the 
researcher’s own coding system. Without repeated, careful looks at the data itself, the analysis is 
buried under multiple layers of structure which only have as much validity as the correspondence 
between the codes and the underlying data. 
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My response to this potential criticism comes on two levels: Ethical and technological 
affordance. Ethically, any researcher following any analysis scheme of any data set has an 
obligation to do so thoroughly. I would concede that NUDIST makes it easier for a researcher to 
fool herself. Still, the ethical obligation mitigates effectively against the creation of fatally flawed 
data analysis structures. 

From a technological affordance view, NUDIST makes it very simple to include the data itself 
into any coding report. In the example given above, when generating a report on how often 
participants expressed concern about access to technology, I could include the text of statements 
given that code, allowing me to re-examine the appropriateness of the coding. 

Study 2: Academic. Social, and Personal Uses of the Internet: Cases of Students From an Urban 

Latino classroom 

Study author: Jeff Kupperman 
Study Description 

This study is based on data from a project where the families of middle school students in an ESL 
(English as a Second Language) science class were given home Internet technology as part of a 
project-based unit on the respiratory system (Fishman, Kupperman & Soloway, 1999). The 
technology, dubbed “NetTV” in the study, was a relatively inexpensive device that when 
connected to a television set and a phone line allowed one to browse the World Wide Web and 
use e-mail. The families, who were Mexican and Puerto Rican immigrants living in an inner-city 
neighborhood, had never had home access to the Internet before. Taking a sociocultural view of 
technology adoption (Bruce & Hogan, 1998; Kaptelinin, 1996; MacKenzie, 1996; Nardi, 1996; 
Schement, 1995), the study focused on the ways that children and their families perceived the 
technology and used it for academic, social, and personal purposes. At the center of the study 
are case studies of four students in three families (two students were siblings). 

Data sources 

Data included structured interviews with 14 students in the third month of the project and 
follow-up interviews with three target families at the end of a year; fieldnotes from the classroom 
and home visits made by project staff throughout the project; electronic logs of Internet use by 
each family; and notes taken from 4 selected days of classroom video. A total of 47 text 
documents, many containing data from multiple days, were imported into the NUDIST project. 

These data were used to construct cases studies that explored use of the technology in three 
categories: “family life,” which included interactions among family members around NetTV; 
“school life,” which included use of NetTV for schoolwork, and “kid life,” which included the 
ways NetTV was used for social and expressive activities. Home use of the Internet by first 
time, “non-mainstream” users is an area where little research has been done (Kupperman, 1999), 
and the case studies aimed to generate broad themes, rather than to test particular theories. 
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Methodological and Analytical Issues 

NUDIST was chosen because of the need to organize and process large amounts of textual data, 
as well as look for patterns across data from different sources. NUDIST supported an iterative 
processes of data display and analysis (Huberman & Miles, 1994) that was partly top-down 
(derived from a theoretical perspective) and partly bottom-up (generated from within the data): 
The research design and research questions led to some initial categories, but categories were also 
generated and refined during repeated passes through the data. For example, interviews were 
stmctured enough that the responses could be pre-processed in a form that allowed NUDIST to 
automatically create a category for each question. Each target student also had an a priori 
category to tag all data pertinent to that student. Other codes were created “on the fly” during 
read-throughs of the data or text searches; these were often combined into larger categories later. 
The three large categories of family life, school life, and kid life were created relatively early, and 
later coding was done with these in mind. 

Because the study was exploratory and broad in scope, it was sufficient to identify an area of 
text that corresponded to a particular theme, as opposed to comparing or counting linguistic 
features of the text. Therefore, no special attention was paid to the size or stmcture of text units; 
text units were defined as the default of one word processor line. 

Also, no special effort was made to make sure the final codes were strictly consistent and 
hierarchical. For this paper, a choice was made to do initial organization of data in NUDIST, but 
to do further organization and synthesis through working directly with narrative drafts of the 
case studies. During the writing of the case studies, the NUDIST data file was used to check 
evidence and provide examples, and some new coding was done at this time. However, as the 
writing progressed, use of NUDIST became focused less on manipulating the index system and 
more for retrieving evidence to support arguments in the writing. This choice was partly due to 
personal work style of the author, but it is also due to the fact that much of the data was useful 
for suggesting themes rather than making rigorous comparisons. For example, there was ample 
evidence that one focus student had a particular interest in a website for Lowrider Magazine and 
viewed it again and again. In another example, classroom video captured a conversation between 
several students about looking at the Playboy web site, which was suggestive of the way some of 
the students viewed the Internet. In this case the students’ conversation highlighted a difference 
between one student who saw the Internet as a way to challenge authority and do grown-up 
things, and another student who saw it more as part of her school work. However, since the data 
did not capture every single conversation between students, it was impossible to say, for 
example, how common these conversations were. It was enough to have this piece of data in the 
NUDIST database, and less important how it fit into the coding tree. 

Impact of multiple-level coding 

An important feature of NUDIST for this study was the ability to code any particular segment 
of text in multiple ways, then call up text segments that represented an intersection of certain 
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codes. In particular, it was possible to code separately by family and by theme, and compare 
themes for each family. For example, all text that pertained to the student Jorge was coded 
“Jorge,” and all text that had to do with the use of e-mail by parents was coded “parent e-mail.” 
To examine how Jorge’s parents used e-mail, it was possible to search for the intersection of 
“Jorge” and "parent e-mail." This flexibility made it easy to add, merge, and refine codes, and 
then quickly examine those codes for each case. As mentioned above, the NUDIST database was 
used to retrieve evidence and check hypotheses during the writing of the case studies, and so it 
was important to be able to do these kinds of searches quickly and easily. 

Study 3: Multiplicative Identities: First Year Undergraduates’ Explorations of Their Own 

Multiple Identities 

Study author; Kristen Boyle-FJeimann 
Study Description 

The purpose of this study is to explore how first year undergraduates describe and think about 
their multiple social identities. The term social identity, drawn from the social justice literature, 
refers to identifying as a member of a particular group based on one’s race, religion, ethnicity, 
gender, sexual orientation, socio-economic class, ability or age (Boyle-FJeimarm, 1997). I 
collected data over the course of one academic year acting as an observer in a course in which all 
of the participants were enrolled. Two interviews were conducted with each participant in the 
first term and two in the second term, culminating in a final interview and questionnaire at the end 
of the academic year. Primary data include four audio-taped transcribed interviews with each of 
the sixteen study participants, responses to a series of three written questionnaires, and 
participants’ written work. In addition, individual and group videos and fieldnotes from 
observations of class sessions provide further information and supporting evidence. Multiple 
data sources provide depth and allow for potential triangulation of emerging themes. 

NUDIST was selected as an analytic tool after the study had been designed and during the 
process of data collection. Thus, the choice to use NUDIST did not impact the study design and 
has no methodological implications for this study. 

As I undertook both issue-focused and case-focused analysis (Weiss, 1994) to examine themes 
across cases as well as to explore the richness and depth of particular cases, I employed NUDIST 
to code and examine data. Once documents are coded and the codes entered into NUDIST, the 
software becomes useful in exploring coding patterns and illustrating themes emerging from the 
data. NUDIST makes it feasible and efficient to do thematic analysis across large quantities of 
data from non-contiguous sources. Thus I was able to expediently consider a variety of ideas and 
themes. The use of data analysis software enhances the scope and efficiency of my analysis. 
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Methodological and Analytical Issues 

As described previously, this is an interpretative study exploring how first year undergraduates 
describe and think about their multiple social identities. Issue-focused and case-focused analysis 
(Weiss, 1994) examine themes across multiple data sources. The study centers around case 
studies of three particular students and thematic, issue-focused analysis across data from all of 
the 16 first year undergraduate students enrolled in the same course. 

NUDIST software was chosen for this study due to the need to process and analyze large 
amounts of data from varied sources (interview transcripts, fieldnotes, participants’ written 
work, and questionnaire responses) and across study participants. I had completed a previous 
project analyzing field notes, interview transcripts, and questionnaire responses using a word 
processing program as a slightly updated mode of cutting and pasting all material into coded 
categories. The piles of paper strewn across my floor were imwieldy in a relatively small study, 
and I knew there had to be a more effective way of managing the large quantity of data collected 
for the current study which includes 64 audio-taped interviews and more than 1,000 pages of 
transcripts. 

Effect of text unit size decisions 

When initially processing text documents, NUDIST requires the researcher to select segments of 
data to be coded as text imits. A unit of text is recognized by the software each time there is a 
hard return within a document, and a text unit is the smallest segment that can be coded in 
NUDIST. While it is possible to combine paragraphs by using line breaks instead of hard 
returns, it is more common to either code at the paragraph level (with a hard return at the end of 
each paragraph) or the line level (with a hard return at the end of each line). Any coding scheme 
requires the researcher to make decisions about parameters, and NUDIST requires this choice be 
made when documents are set up to be imported into the software so that the hard returns are 
properly placed to indicate text imits. 

I did not want to make arbitrary or inconsistent decisions about where a particular unit of data 
began and ended and did not desire the level of detail associated with coding each word or line of 
data. Thus, for consistency I decided coding units would be speech units within each interview. 
That is, a coded unit began each time a speaker started to speak and ended each time the same 
speaker stopped speaking. For the participants’ written work, paragraphs were used as coding 
units to honor where the participant had decided to begin and end a particular set of ideas. 

Coding in NUDIST then had to be conducted with these as parameters for text units. 

Coding at the paragraph and speech unit was appropriate for the analysis needs of this study. 
Speech units generally followed a common theme. In rare cases participants switch topics 
dramatically within a speech unit. In those cases, multiply coding a paragraph allowed the text to 
be examined from both topics. As I knew I was not doing linguistic analysis or the kind of 
quantitative/qualitative analysis done in study one, coding at the single line level was not 
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necessary. Coding at the paragraph level honored participants’ sense of where the speech broke 
while giving a fine enough grain to the text for the analysis purposes. 

Iterative nature of coding process 

The coding scheme for this study began with an initial line-by-line reading of the entire corpus of 
data on 1 6 first year undergraduates collected over the course of one academic year (64 interview 
transcripts, 48 sets of questionnaire responses, students’ papers from first term, fieldnotes from 
throughout the academic year, and notes summarizing participants’ individual and group videos). 
This initial reading of the data identified themes, patterns, and variations in the data and led to a 
preliminary set of approximately 550 inductively generated codes (Emerson, Fretz, and Shaw, 
1995). The initial set of codes was examined for duplications, common ideas that could be linked 
under one code, and codes that were too specific or narrow to be broadly useful. This iterative 
process led to a refined list of approximately 370 codes and subcodes. The refined set of codes 
was employed to code pilot data from two study participants not selected as case studies. After 
the pilot coding, I again refined the codes, eliminating duplication and codes that were too specific 
or narrow (e.g., codes generated by or used only in one small chunk of data), leading to 
approximately 300 codes and subcodes. As the coding scheme developed, NUDIST provided the 
flexibility to refine codes by deleting, renaming, and combining codes. These codes were then 
used to code data from participants selected as case studies. During the process of developing 
and refining codes, I kept lists of the codes in a Microsoft Word file, using Word to alphabetize 
the codes, with subcodes indexed under the main code. In addition, during this process I also 
wrote memos tracing my thinking and rationale (Emerson, Fretz, and Shaw). 

Initial coding identified many more themes than could be pursued in one project. Thus, I then 
focused analysis for each case study on recurrent issues linked to what had emerged as a focal 
interest in the study (Weiss, 1 994) — how participants talk about and describe their struggles 
around social identities. Codes and coded data had been entered in NUDIST, and now the 
software facilitated and expedited the process of focused coding and analysis by allowing me to 
produce and print reports displaying data coded around the focal themes identified. I read and 
analyzed the NUDIST-generated reports to distinguish subthemes, subtopics, differences and 
variations within the central topics in a particular case study (Emerson, Fretz, and Shaw, 1995; 
Stake, 1995; Weiss, 1994). 

For example, focused coding of the case of the same male participant previously described might 
lead to pursuing issues around his description of gender issues in his experiences. Analysis of a 
report on all data for this case coded “gender” might highlight the theme of this participant’s 
relationships with women as a potentially fruitful area for further analysis. This theme would 
then be elaborated and explored as the case study is written. 

Next, an initial set of guiding questions focused coding of data from the three case studies around 
issues such as how participants talk about their own multiple identities. This initial focus was 
broad enough to permit exploration of a wide variation of themes accented in each case. 

Therefore, it was necessary to employ a large number of codes and subcodes. NUDIST 
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facilitated the use of a large number of codes because the software made it possible to effectively 
manage and display data coded with numerous codes. The flexibility to manage and utilize a large 
number of codes is thus a critical affordance of NUDIST allowing the researcher to identify and 
explore a wide variety of themes and patterns. 

1 used NUDIST to code, sort, select, and display data. Coded data was then employed in 
thematic analysis both within and across cases (Weiss, 1994). During the processes of 
developing initial guiding questions for the study and then collecting data, 1 wrote memos to track 
the evolution of my thinking around the ideas, themes, and issues and recorded my insights on 
topics and categories identified during data collection as areas of initial interest and areas to 
pursue further (Emerson, Fretz, and Shaw, 1995). Initial data analysis followed the patterns of 
open and focused coding suggested by Emerson, Fretz, and Shaw. First, after all of the data had 
been collected, 1 read through the entire corpus of data collected line by line to elaborate and 
expand themes and patterns identified in memos and to identify any new themes not previously 
recognized. 1 then employed open coding, reading interview transcripts, participants' written 
work, fleldnotes, and notes on participant-produced videos, and continued to identify and 
formulate themes and ideas to both distinguish new and disparate themes and to explore patterns 
and themes cutting across data. 

Analysis affordances and constraints 

A particularly useful feature of NUDIST’ s numerous options in how to sort coded data for 
analysis is the ability to code an entire document with “base data” and “case data” codes. Base 
data, referred to as structural codes in study one above, in this study included demographic 
information about the participants (gender, race, religion, etc.) and the type of data (interview, 
participants’ written work, etc.). This permits examination of a particular category of data. For 
example, 1 might wish to examine what all of the Jewish women in the study said about religion 
by examining data that was coded with base codes “female” and “Jewish” and the specific code 
“religion.” It is quite expedient to explore data from varied data sources (e.g., interview 
transcripts, participants’ papers, and fieldnotes) and from varied participants. It is also possible 
to explore alternative themes so that analysis is not restricted by time limitations in the same 
way it might be with more traditional methods of data organization and processing. The 
exploration of alternative themes allowed me to evaluate which themes identified by coding and 
analysis were fruitful for further analysis and which were not important or relevant to the 
particular study at hand. 

For example, in one case study the participant ranked sexual orientation as an extremely 
important identity in the questionnaire responses, yet my sense from coding the interview 
transcripts and writings for this participant was that sexual orientation rarely came up other than 
in the questionnaire. To explore to what extent and in what ways the participant talked about 
sexual orientation in interviews and written papers, 1 generated a report in NUDIST of all 
excerpts from this participant’s documents coded as “sexual orientation.” The report supported 
my hunch that the participant rarely mentioned sexual orientation. It may be relevant to point to 
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and comment on this finding in writing up my analysis, but the finding also suggested that there 
was little data I could use to directly explore the issue of sexual orientation in this case study. 

The reporting capacities of NUDIST again provide flexibility. Numerous codes can be combined 
in one report to explore a particular theme. For example, to explore a male participant's 
relationships with women, I might look at all of the data coded for this case with the codes 
“gender,” “friends - gender and friends,” “romance,” and “social life.” Just examining the data 
coded with one of those codes would not capture the scope of data I desire. If I later decide that 
it would also be useful to look at this participant’s relationship with his mother in exploring his 
relationships with women, it would be easy to simply add the code “family - relationship with 
parents” and generate a new report. I am not hesitant to explore or try new themes and ideas 
because it is easy and quick to gather and display the relevant coded data. Ultimately, the 
flexibility and expediency NUDIST allows may afford a more thorough, comprehensive, and 
sophisticated analysis than would be feasible with traditional methods. 

Efficiency of software use 

As with all research methods and tools, the use of NUDIST brings not only benefits but 
constraints. For example, a colleague pointed to a potential constraint in using NUDIST in that 
the researcher does not have to go through the process of sorting the data by hand (or by cutting 
and pasting data in a word processing program) into categories. I have been asked if I am missing 
something by using software to sort and collate data for me. 

The decision about what data to sort and how to sort it is still mine. As noted in the example 
above, in order to explore the participant’s relationships with women I had to make decisions 
about what type of data and what codes to examine. In addition, I actually found my analysis to 
be more effective than when I used a more traditional method for data sorting and processing in a 
previous study because I could spend my time exploring varied themes rather than spending 
countless hours sorting data. 

Technological capabilities and opportunities as an analytical issue 

Finally, although the current paper focuses on conceptual rather than technical issues in the use 
of software to aid in data analysis, it is important to note that the researcher’s familiarity and 
comfort level with technology and software may well impact how extensively she employs the 
varied modes through which NUDIST can be utilized. The particular affordances and constraints 
I have discovered using NUDIST may well differ in part from those of the other two authors of 
this paper due to the differences in our expertise utilizing technology. Kupperman and 
Margerum-Leys work on research projects and in capacities specifically designed around uses of 
technology. In addition, they work in a special office area that purposefully provides access to 
advanced hardware and software. In contrast, I work at home on a relatively simple computer 
with limited software, not in a technology laboratory. I am not a “techie” as are my two 
colleagues. Our exposure to and comfort level with software such as NUDIST varies greatly. 
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The implications of our varied expertise and comfort with technology are that my use of 
NUDIST is largely confined to using the software in a basic manner to do the type of analysis I 
would have without the software but in a more expedient and thus more extensive manner, as 
described above. In contrast to my use of NUDIST to largely make analysis easier and more 
efficient, the other two study authors are able to use the software to expand the types of 
analyses they do and can explore the limits of what the software can do to meet their needs. 

Conclusion 

The three studies which comprise this paper are quite different from each other in area of 
interest, theoretical frame, methodology, scope, and analysis priorities and methods. These 
differences allow us to consider multiple perspectives on our common use of NUDIST for data 
analysis purposes. 

Study one draws for its theoretical frame on the literature surrounding teacher beliefs. 
Methodologically, it is an interview-based beliefs study. As such, its analysis does not make any 
claims to describing reality; rather, it seeks to describe participants’ perspectives and 
commentary on the role of technology in teaching and learning. 

The core of study two is a set of case studies which draw from a variety of data sources related 
to home use of the Internet by three families. Interviews, field notes, video, and computer log 
files were used in combination to create the case studies. While each data source by itself 
provides an incomplete picture of the families' Internet use, together they provide enough 
evidence to identify themes for each family. 

Study three explores first year undergraduates’ descriptions and thinking about their own 
multiple social identities. Methodologically, it is a multi-method interpretive study centering 
largely around interview transcripts, participants’ written coursework, and questionnaire 
responses. It focuses mainly on social identities but also attends to interlinking social and 
personal identities acknowledging the ways in which identity is both “personal (residing within 
the individual) and social (residing within social relationships and society)” (Ashmore and 
Jussim, 1997, p. vii). 

Table 1 below gives an overview of the methodological and analytical issues addressed by each of 
the study authors. 
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Table 1: Issues Addressed 



Margerum-Leys: Teacher education students’ 
beliefs about technology 


• Analysis capabilities as an influence on 
design 

• Calling attention to data trends 

• Justifying the reporting of data trends 

• Making the data structure explicit 

• Possibility of over-rating the validity of the 
structure 


Kupperman: Academic, social, and personal 
uses of the internet 


• Impact of multiple-level coding 


Boyle-Heimami: Multiplicative identities 


® Effect of text unit size decisions 

• Iterative nature of coding process 

• Analysis affordances and constraints 

• Efficiency of software use 

• Technological capabilities as an analytical 
issue 



The issues addressed above are primarily analytical in nature, as we have defined ‘analytical’ for 
this paper’s purposes. As NUDIST is an analytical tool, it is not surprising that it is in the 
analysis phase that we perceive NUDIST to have the most impact. However, these analytical 
themes in turn impact study design and ultimately have an effect on the reporting and writing 
phases of research as well. Seen in aggregate, the issues raised in the studies above give a range of 
examples of the ways in which the use of qualitative data analysis software (in this case, 
NUDIST) impacts qualitative research in ways that go beyond technological considerations. 

Educational Importance 

By providing perspectives gained from three studies which represent a breadth of theoretical 
frames and data types while employing a common data analysis software, we address analytical 
and methodological issues raised through the use of this software. Affordances of the software 
influenced both the methodologies and analysis schemes used by the researchers. The ability to 
give a concrete structure to large data sets, code at multiple levels, and pursue iterative analysis 
methods have repercussions for multiple phases of the research endeavor. 

As researchers consider research design and the role of data analysis software, we hope that the 
methodological implications of the use of such software are weighed alongside the promise of 
technical efficiency. While the impact of technology may be more pragmatic than theoretical, the 
conceptual issues outlined in this paper merit consideration when making methodological and 
analytical decisions regarding research design, enactment, analysis, and reporting. 
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Appendix A: Coding Structure for Study 1 



The coding maps below were created by QSR NUD*ST and Inspiration Software’s Inspiration. 
Since the entire map is too large to fit on a single page, 1 begin with a figure showing the major 
categories, followed by a separate figure showing each category and its associated codes. 



For figures two through nine, the category Figures, the following convention is used: The main 



topic area for the category is denoted by a darkly shaded rectangle Subcategories are 

denoted by an lightly shaded rectangle Creation of a subcategory occurred when a 

group of related codes was grouped. For instance, in category 1, “Background Information”, 
separate subcategories were used to group background regarding technology and background 

outside of technology. Codes themselves are indicated by a shaded oval (O). 
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Figure 7: Background Information. 12 Codes 
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Figure 8: Category 2. Social Dynamics of Computers. 14 Codes 
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Figure 9: Category 3. Computers as Used bv Teachers. 7 Codes 
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Figure 10: Category 4. Computers and Curriculum. 1 1 Codes 
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Figure 11: Category 5. Computers as Used bv Students. 13 Codes 
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Figure 12: Category 6. Learning About Computers tParticipants'), 10 Codes 
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Figure 13: Category 7. Knowledge About Computers. 12 Codes 
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Figure 14: Category 8. Availability of Computers. 4 Codes 




Analytical and methodological issues in qualitative data analysis software 34 



Appendix B: Coding Scheme for Study 2 

Coding by student: 

•Jorge 

•Estella 

•Alicia 

•Manuel 

Coding by theme: 

•Kid life 

0 Testing limits 
0 Student-student email 
0 Tech skill as social capital 
0 Peer networking 
0 Inappropriate 
0 Student surfing 
• Family life 

0 Anxiety 

0 Family (non) use of tech 
0 Parent email 
0 Parent participation 
0 Parent-child interaction around NetTV 
0 Parent wishes about NetTV 
0 Student leading 
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Analytical and methodological issues'in quSlitative data analysis software 

0 Push for parent participation 

• School life 

0 Web for Homework 
0 Educational Attitudes 
0 T to SS about NetTV 
0 Other computer use 
0 Email for homework 
0 Privileging NetTV use, skill 
0 Academic use of NetTV 
0 Rationale for NetTV 

• Value of NetTV to teacher 
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